A hybrid MPI-OpenMP scheme for scalable parallel pseudospectral computations for fluid turbulence

摘要

A hybrid scheme that utilizes MPI for distributed memory parallelism andOpenMP for shared memory parallelism is presented. The work is motivated by thedesire to achieve exceptionally high Reynolds numbers in pseudospectralcomputations of fluid turbulence on emerging petascale, high core-count,massively parallel processing systems. The hybrid implementation derives fromand augments a well-tested scalable MPI-parallelized pseudospectral code. Thehybrid paradigm leads to a new picture for the domain decomposition of thepseudospectral grids, which is helpful in understanding, among other things,the 3D transpose of the global data that is necessary for the parallel fastFourier transforms that are the central component of the numericaldiscretizations. Details of the hybrid implementation are provided, andperformance tests illustrate the utility of the method. It is shown that thehybrid scheme achieves near ideal scalability up to ~20000 compute cores with amaximum mean efficiency of 83%. Data are presented that demonstrate how tochoose the optimal number of MPI processes and OpenMP threads in order tooptimize code performance on two different platforms.